Methods for Life-long Planning and Failure Detection in Dynamic Environments

It is essential for successful human-robot interaction that the robot is able to both qualitatively understand the intended meaning of the human’s utterances and to react to them in the continuous real world. In this paper we present a number of novel techniques that collectively allow a robot to interpret and execute spoken commands describing manipulation goals involving qualitative spatial constraints (e.g. “put the red ball near the blue cube”). The resulting implemented system integrates computer vision, potential field models of spatial relationships, and action planning to mediate between the continuous real world, and discrete, qualitative representations used for symbolic reasoning.